ACCELERATED LEARNING BY ACTIVE EXAMPLE SELECTIONByoung

Much previous work on training multilayer neural networks has attempted to speed up the back-propagation algorithm using more sophisticated weight modiica-tion rules, whereby all the given training examples are used in a random or predetermined sequence. In this paper we investigate an alternative approach in which the learning proceeds on an increasing number of selected training examples, starting with a small training set. We derive a measure of criticality of examples and present an incremental learning algorithm that uses this measure to select a critical subset of given examples for solving the particular task. Our experimental results suggest that the method can signiicantly improve training speed and generalization performance in many real applications of neural networks. This method can be used in conjunction with other variations of gradient descent algorithms.